Time of flight imaging is a type of depth sensing technology used in many computer vision applications such as object tracking and recognition, human activity analysis, hand gesture analysis, and indoor 3D mapping, amongst others.
A time of flight system comprises one or more light sources which emit rays of light into a scene, and a light sensor such as a camera. A time of flight system works by computing the time, measured as a phase shift, it takes a ray of emitted light to bounce off a surface and return to a camera at the system. This gives a measurement of the depth of the surface from the camera. Time of flight systems are generally able to achieve reasonable accuracy, and, where they use light in the infrared spectrum, to operate in low illumination settings.
However, time of flight systems suffer from multipath interference (henceforth simply “multipath”). Where the emitted rays of light are sent out for each pixel, and since light can reflect off surfaces in myriad ways, a particular pixel may receive photons originally sent out for other pixels as well. This results in corrupted sensor measurements. These corruptions do not look like ordinary noise, and can be quite large, resulting in highly inaccurate depth estimates. For example, significant multipath is observed in scenes with shiny or specular-like floors.
Removing the effect of multipath is therefore a crucial component to enable accurate time of flight systems. Attempts to remove the effect of multipath have involved using additional sensors. Other approaches have used radiometric models; or radiometric and photometric models. There is an ongoing need to improve the accuracy of time of flight systems by improving multipath removal. However, for many practical applications such as object tracking, hand gesture recognition and others, the time of flight system needs to be both accurate and fast so that accurate depth measurements are output in real time, for example, at a frame rate of a camera capturing a stream of images of the scene.
The embodiments described below are not limited to implementations which solve any or all of the disadvantages of known time of flight depth measurement correction systems.